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THE REPLY FILED 01 October 2008 FAILS TO PLACE THIS APPLICATION IN CONDITION FOR ALLOWANCE. 

1 . £3 The reply was filed after a final rejection, but prior to or on the same day as filing a Notice of Appeal. To avoid abandonment of this 

application, applicant must timely file one of the following replies: (1 ) an amendment, affidavit, or other evidence, which places the 
application in condition for allowance; (2) a Notice of Appeal (with appeal fee) in compliance with 37 CFR 41 .31 ; or (3) a Request 
for Continued Examination (RCE) in compliance with 37 CFR 1.114. The reply must be filed within one of the following time 
periods: 

a) CD The period for reply expires months from the mailing date of the final rejection. 

b) K| The period for reply expires on: (1) the mailing date of this Advisory Action, or (2) the date set forth in the final rejection, whichever is later. In 

no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of the final rejection. 

Examiner Note: If box 1 is checked, check either box (a) or (b). ONLY CHECK BOX (b) WHEN THE FIRST REPLY WAS FILED WITHIN TWO 

MONTHS OF THE FINAL REJECTION. See MPEP 706.07(f). 
Extensions of time may be obtained under 37 CFR 1 .136(a). The date on which the petition under 37 CFR 1.136(a) and the appropriate extension fee 
have been filed is the date for purposes of determining the period of extension and the corresponding amount of the fee. The appropriate extension fee 
under 37 CFR 1.17(a) is calculated from: (1) the expiration date of the shortened statutory period for reply originally set in the final Office action; or (2) as 
set forth in (b) above, if checked. Any reply received by the Office later than three months after the mailing date of the final rejection, even if timely filed, 
may reduce any earned patent term adjustment. See 37 CFR 1 .704(b). 
NOTICE OF APPEAL 

2. □ The Notice of Appeal was filed on . A brief in compliance with 37 CFR 41 .37 must be filed within two months of the date of 

filing the Notice of Appeal (37 CFR 41 .37(a)), or any extension thereof (37 CFR 41 .37(e)), to avoid dismissal of the appeal. Since a 
Notice of Appeal has been filed, any reply must be filed within the time period set forth in 37 CFR 41.37(a). 
AMENDMENTS 

3. □ The proposed amendment(s) filed after a final rejection, but prior to the date of filing a brief, will not be entered because 

(a) HH They raise new issues that would require further consideration and/or search (see NOTE below); 

(b) \Z\ They raise the issue of new matter (see NOTE below); 

(c) □ They are not deemed to place the application in better form for appeal by materially reducing or simplifying the issues for 

appeal; and/or 

(d) Q They present additional claims without canceling a corresponding number of finally rejected claims. 

NOTE: . (See 37 CFR 1.1 16 and 41.33(a)). 

4. □ The amendments are not in compliance with 37 CFR 1.121. See attached Notice of Non-Compliant Amendment (PTOL-324). 

5. n Applicant's reply has overcome the following rejection(s): . 

6. □ Newly proposed or amended claim(s) would be allowable if submitted in a separate, timely filed amendment canceling the 

non-allowable claim(s). 

7. □ For purposes of appeal, the proposed amendment(s): a) □ will not be entered, or b) □ will be entered and an explanation of 

how the new or amended claims would be rejected is provided below or appended. 
The status of the claim(s) is (or will be) as follows: 

Claim(s) allowed: . 

Claim(s) objected to: . 

Claim(s) rejected: . 

Claim(s) withdrawn from consideration: . 

AFFIDAVIT OR OTHER EVIDENCE 

8. □ The affidavit or other evidence filed after a final action, but before or on the date of filing a Notice of Appeal will not be entered 

because applicant failed to provide a showing of good and sufficient reasons why the affidavit or other evidence is necessary and 
was not earlier presented. See 37 CFR 1.116(e). 

9. □ The affidavit or other evidence filed after the date of filing a Notice of Appeal, but prior to the date of filing a brief, will not be 

entered because the affidavit or other evidence failed to overcome all rejections under appeal and/or appellant fails to provide a 
showing a good and sufficient reasons why it is necessary and was not earlier presented. See 37 CFR 41 .33(d)(1 ). 

1 0. □ The affidavit or other evidence is entered. An explanation of the status of the claims after entry is below or attached. 
REQUEST FOR RECONSIDERATION/OTHER 

11. The request for reconsideration has been considered but does NOT place the application in condition for allowance because: 
See Continuation Sheet. 

12. □ Note the attached Information Disclosure Statements). (PTO/SB/08) Paper No(s). 

13. □ Other: . 

/Richemond Dorvil/ 

Supervisory Patent Examiner, Art Unit 2626 
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Continuation of 1 1 . does NOT place the application in condition for allowance because: Re Remarks, Examiner takes the position that 
both Van Thong and Bloom in fact clearly teach the limitations as described in the claims, wherein Van Thong teaches a 
text/script/discourse/sentential alignment module that is able to handle a plurality of text and speech recordings, where a user can navigate 
selections. Van Thong teaches a first module, the audio classifier 15, sorts the input audio 13 into different categories: spoken text, music, 
etc. Of interest are in the spoken parts of the input audio 13 track because the spoken parts need to be transcribed. Possibly, a particular 
noise or sound other than spoken language may need to be captioned. However, only the spoken parts 1 7 as sorted or filtered by the 
audio classifier 15 are sent to the next module 19. A third module, the time event tracker 23 receives the time-stamped audio 21 and 
records the time the words were typed in by the operator 53. This provides a rough time alignment of the corresponding text 25 that will be 
precisely realigned by the next module 29. The recorded time events are mapped back to the original time scale. Thus the time event 
tracker 23 produces on output roughly aligned transcription text 27. A fourth module 29 receives the roughly aligned text 27 and realigns 
precisely the text on the audio track 13 using speech recognition techniques at the word level. Realigner 29 thus outputs aligned 
transcribed text 31 (Col. 3 line 41 - Col. 4 line 5 & Fig. 1). 

Further, Van Thong teaches user retrieval of text, wherein Van Thong teaches the end user is thus presented with a visual display of text 
that corresponds to the retrieved audio. Upon playback of the audio (in response to the user selecting or otherwise issuing a "play" 
command), the sound system of the user's computer produces the subject audio track while the screen displays the produced transcription 
text in synchronization with the audio. In a preferred embodiment, the audio that is downloaded and processed by the speech recognition 
module is deleted after the transcription process. A pointer to the server where the audio is stored is embedded in the displayed results. 
The embedded pointer is coupled to the "play" command to effect retrieval and rendering of the audio upon user command (Col. 8 lines 18- 
30). 

Furthermore, Van Thong teaches technological limitations of a case where a multimedia data stream contains an audio channel, speech 
technology can be used to correlate the data stream with a text transcript. By performing speech recognition on the audio channel it may 
be possible to correlate words in a transcript with positions in the data stream. However, this approach is limited to data streams with 
speech audio on which speech analysis is successful. Van Thong overcomes these limitations by teaching an interactive user multimedia 
system, wherein Van Thong teaches that given a position in a particular data stream, the closest data pointer stored in the index can be 
identified. The time-stamp for that entry can then be compared to the time-stamps for the other data streams, resulting in a set of 
corresponding entries in the index. The data pointers for these entries can be followed, given the corresponding positions in the other data 
streams. For example, suppose that p.sub.O was the closest data pointer to a given query position. Then since t.sub.O is matched to 
t'.sub.O (after appropriate event filtering and data pointer interpolation), it follows that q.sub.O is the position in the second data stream that 
corresponds to the query position. This could be used in the transcription example to synchronize the display of the transcript 79 with the 
playback of the audio 89. It could also be used for multimedia indexing. In that case, text queries would produce matches in the transcript 
text 79 which could then be linked directly to segments of audio 89 content (Col. 18 lines 5-21 & fig. 1). 

Bloom has been incorporated to further strengthen the teachings of Van Thong, wherein Bloom like Van Thong teaches text and time 
alignment abilities in a media system. Bloom teaches a graphical user interface 320 as it appears on the display screen 130 in this 
embodiment is shown in FIG. 3. The interface 320 is divided into five main areas: a Scene display and selection area 340; a main video 
display window 330; a Line display and selection window 350; Line operation buttons 361 through 363; a Stop button 364 and a Revoiced 
Take display and selector or Recorder section 370, which is labeled "Recorder" for the end user. Only the main video display window 330 
does not serve as a control. The Recorder section 370 indicates the available user Revoiced Takes, such as 371 , 373 and 374 and the 
Original Recording 372. The pointing device 120 is used to position a cursor (not shown) on and to select and operate (e.g. if a mouse, by 
clicking) any of the controls shown in ways to be described below (Bloom [0091] & fig. 3). 

Further, Bloom teaches user interaction with multiple recordings and alignment/synchronization, wherein Bloom teaches the Aligned audio 
is stored on the hard disk 140 (or in RAM 118) and its existence and identity will be displayed in one of the previously "empty" Take 
recording holders 371 , 373, 374 in the Recorder section 370. As shown in FIG. 3, for example, the user would see a name in the 
position of the selected Recording holder 371, labeled here "Recording M+1". In fact, only data pointing to the location of the Aligned audio 
on the disk 140 (or in RAM) will be held. The user can, optionally, generate commands via the pointing device 120 to audition, "erase" or 
name any of these recordings by using, for example, a right click menu if the device is 120 is a mouse. A name is entered by use of the 
keyboard 125 in a naming mode selected by the device 120. (Bloom [0100] & fig. 3). 

Furthermore, Bloom teaches text alignment, audio alignment, user interaction, and editing, wherein Bloom teaches the alignment of a user's 
recording, where an automatic audio alignment algorithm is used which analyses both audio signals and automatically edits a Temporary 
User's recording 856 in FIG. 8 (one of which will be referred to herein as the "Dub" audio because that is a known term in the audio-post 
production industry) to align to the Guide audio. Such an algorithm has been described in GB2117168 and U.S. Pat. No. 4,591,928 
(Bloom et al). It comprises the essential steps of: (a) measuring the same time-varying features of both the Guide and Dub audio and 
saving these measurements as digita feature data; (b) processing the resulting time-varying feature data to determine a time distortion path 
that best aligns the Dub audio features to the Guide audio features; and (c) passing the time distortion path to an automatic waveform 
editor which edits the waveform of the Dub audio according to the time distortion path, whilst also taking into account the properties of the 
waveform data of the Dub audio in order that edits made in the new aligned waveform do not introduce audible artifacts (Bloom [0193]). 

Additionally, Bloom teaches that it is well known in the art to use an audio alignment algorithm, wherein Bloom teaches that it has been 
assumed that during the Recording phase the user reads and records the words in the script correctly, but does not achieve perfect sync 
with the Original Character's voice in the original dialog audio and the corresponding lip movements. It is also assumed that, in the 
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event the user makes a substantial error in reading the text, the user will stop and make further attempts until he gets it essentially correct 
apart from timing. If, however, there is an error in the reading, or the user records words different from those in the script, and the audio 
alignment algorithm achieves results similar to that aimed for in the algorithm in U.S. Pat. No. 4,591 ,928 (Bloom [0207]).. 
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